Papers with commonsense QA
CALM-Bench: A Multi-task Benchmark for Evaluating Causality-Aware Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in foundation language models have shown the efficacy of pre-trained models across diverse QA tasks. |
| Approach: | They propose a multi-task benchmark for evaluating causality-aware language models to unify causal QA research. |
| Outcome: | The proposed model outperforms single-task fine-tuned models on the CALM-Bench tasks. |
Teaching LLMs to Abstain across Languages via Multilingual Feedback (2024.emnlp-main)
Copied to clipboard
Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Orevaoghene Ahia, Shuyue Stella Li, Vidhisha Balachandran, Sunayana Sitaram, Yulia Tsvetkov
| Challenge: | Existing studies on LLM abstention focus on English, but they show that it can reduce the accuracy of the model by 20.5% . |
| Approach: | They propose to teach LLMs to abstain in the face of knowledge gaps by generating multiple feedback items in related languages. |
| Outcome: | Extensive experiments show that the proposed approach outperforms baselines and achieves 9.2% improvement for low-resource languages. |
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on logical reasoning tasks; it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts. |
| Approach: | They perform a controlled evaluation of zero-shot CoT reasoning in two socially sensitive domains: harmful questions and stereotype benchmarks. |
| Outcome: | The results show that zero-shot CoT reasoning increases model’s likelihood to produce harmful or undesirable output, but decreases with improved instruction following. |
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference (2023.acl-long)
Copied to clipboard
Junhao Zheng, Qianli Ma, Shengjie Qiu, Yue Wu, Peitian Ma, Junlong Liu, Huawen Feng, Xichen Shang, Haibin Chen
| Challenge: | Existing studies attribute catastrophic forgetting to fine-tuning, and they retain pre-trained knowledge indiscriminately without identifying what knowledge is transferable. |
| Approach: | They propose a unified objective for fine-tuning to retrieve the causality back from pre-trained data and use it to mitigate negative transfer while preserving knowledge. |
| Outcome: | The proposed method outperforms state-of-the-art fine-tuning methods on commonsense QA datasets and can be implemented as a plug-in module to inflate the performance of existing QA models. |
LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for implementing large language models are limited by high computational and memory requirements. |
| Approach: | They propose a lightweight binarization framework that achieves effective W(1+1)A4 quantization through a novel three-stage quantization strategy. |
| Outcome: | The proposed framework surpasses state-of-the-art methods on W2A4 quantization settings across languages. |